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IMAGE PROCESSING APPARATUS AND METHOD 
AND STORING MEDIUM 



G 



BACKGROUND OF THE INVENTION ^flfe?/)/ " f <?Oni 

Field of the Invention er ^6Q 

The invention relates to image processing 
apparatus and method for efficiently compressing an 
image and to a storing medium in which such a method 
has been stored . 
Related Background Art 

In recent years, computers and networks have 
remarkably been developed and various information such 
as character data, image data, audio data, and the like 
are handled in a computer and in a network. 

With respect to the image data and audio data, 
data of a relatively large data amount is often 
handled. Therefore, a process to reduce the data 
amount by compressing the image data and audio data has 
been performed hitherto. 

For example, by compressing the image data, a 
large quantity of image information can be transmitted 
through a network at a high speed. 

According to the above image compressing 
technique, a system (generally called a JPEG) for 
compressing a multivalue still image in the ITU-T 
Recommendation T.81 is known and a system (generally 
called a JBIG) for compressing a binary still image in 
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the ITU-T Recommendation T.82 is known. 

The JPEG system is suitable for compression of a 
natural image such as a photograph* The JBIG system is 
suitable for compression of a binary image such as a 
5 black and white character. 

According to the conventional JPEG system and JBIG 
system, all images of mult lvalue images (photograph or 
the like) and binary images (character image, diagram 
image, or the like) cannot be efficiently compressed 

10 without deteriorating a picture quality. 

To solve the above problem, a method of separating 
a multivalue image portion and a binary image portion 
in one picture plane and compressing them by different 
compressing systems is also known. 

15 The JPEG system is suitable for encoding in a case 

where natures of images locally differ. By properly 
switching an encoding system every area in each block, 
an efficient encoding can be executed while suppressing 
a deterioration of the picture quality. According to 

20 the above system, after multivalue image data was 

converted into a frequency component every block, by 
entropy encoding the frequency component, even if the 
natures of the images locally differ, the encoding 
according to each block can be executed. 

25 On the other hand, the JBIG system does not 

execute the encoding on a block unit basis. 

However, the system for compressing the multivalue 
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image and binary image as mentioned above does not 
satisfy all of the requirements of the user yet. 
Therefore, a new image compressing system is proposed 
to further improve a picture quality, a processing 
5 speed, or an encoding efficiency. 

SUMMARY OF THE INVENTION 

The invention is made in consideration of the 
above conventional systems and it is a main object of 
10 the invention to provide a new system which can 
efficiently compress and can keep a high picture 
quality for any one of a mult lvalue image and a binary 
image . 

Particularly, another object of the invention is 
15 to provide an encoding system of a high compression 

ratio in an encoding system for encoding an image every 
block. 

To accomplish the above objects, according to one 
preferred embodiment of the invention, there is 
20 provided an image processing apparatus comprising: 

first converting means for generating conversion 
coefficients of M values by converting input image data 
by a first system; 

second converting means for generating conversion 
25 coefficients of N values (M > N) by converting the 
input image data by a second system; 

input means for selectively inputting the 
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conversion coefficients of M values or the conversion 
coefficients of N values; and 

output means for outputting first information 
indicating a position where a significant conversion 
5 coefficient exists in a block constructed by either 
ones of the conversion coefficients inputted by the 
input means and second information indicating the 
significant conversion coefficient. 

According to the invention, still another object 
10 of the invention is to efficiently entropy encoding 
block division information which is generated by a 
block division processing as an element technique of 
the new image encoding system. 

To accomplish the above object, according to one 
15 preferred embodiment of the invention, there is 

provided an image processing apparatus comprising: 

output means for outputting division information 
indicating whether each of a plurality of blocks 
including an encoding subject block has further been 
20 divided into a plurality of blocks or not; and 

entropy encoding means for entropy encoding the 
division information corresponding to the encoding 
subject block on the basis of a presumption probability 
according to a division situation of a neighboring 
25 block of the encoding subject block. 

The above and other objects and features of the 
present invention will become apparent from the 
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following detailed description and the appended claims 
with reference to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
5 Fig. 1 is a whole circuit diagram of an image 

processing apparatus on the encoding side; 

Fig. 2 is a diagram for explaining a block 
dividing method of a block division unit 7; 

Fig. 3 is an outline explanatory diagram of a 
10 discrimination unit 1; 

Figs. 4A and 4B are outline explanatory diagrams 
of a discrete wavelet conversion unit 2; 

Figs. 5A, 5B, 5C, 5D and 5E are outline 
explanatory diagrams of an area dividing unit 4; 
15 Figs. 6A and 6B are outline explanatory diagrams 

of a prediction conversion unit 3; 

Figs. 7A, 7B, 7C, 7D, 7E and 7F are diagrams 
showing a data format of encoding data; 

Fig. 8 is a whole circuit diagram of an image 
20 processing apparatus on the decoding side; 

Figs. 9A, 9B, 9C, 9D, 9E and 9F are diagrams 
showing a state of an image decoding on the decoding 
side; 

Fig. 10 is an explanatory diagram of progressive 
25 encoding and decoding of a multivalue image; 

Figs. 11A, 11B and 11C are explanatory diagrams of 
a progressive encoding of a binary image; 
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Figs. 12A, 12B, 12C and 12D are explanatory 
diagrams of a progressive decoding of a binary image; 

Fig. 13 is a block diagram according -to the third 
embodiment ; 

5 Fig. 14 is a diagram sowing a flow of processes of 

a block dividing circuit 103; 

Figs. 15A and 15B are diagrams showing an example 
of a subject block and its dividing state; 

Fig. 16 is a diagram showing a bit string for an 
10 example of the subject block shown in Figs. 15A and 
15B; 

Fig. 17 is a diagram showing a block diagram in 
the fourth embodiment according to the invention; 

Figs. 18A, 18B and 18C are outline diagrams of a 
15 wavelet conversion by a discrete wavelet conversion 
circuit 503; 

Fig. 19 is a diagram showing a flow for processes 
of a block dividing circuit 505; and 

Fig. 20 is a diagram showing a state where a block 
20 is further divided by the block dividing circuit 505. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 1 is a schematic diagram of an image 

processing apparatus which is used in the first 
25 embodiment of the invention. 

A flow of processes of each unit will be first 

simply explained. 
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In the diagram, image data which is inputted from 
an input unit in the diagram is multivalue image data 
having a predetermined number of bits per pixel. 

In the embodiment, it is assumed that not only 
5 image data showing a photograph, a picture, or the like 
suitable for multivalue expression but also image data 
showing characters, diagram, or the like to be 
inherently expressed by a binary value is included in 
the multivalue image data having a depth of 8 bits per 
10 pixel. 

In the embodiment, therefore, to efficiently 
encode image data showing an image in which those 
images mixedly exist, the former image is encoded as 
multivalue image data and the latter image is encoded 
15 as binary image data, thereby improving an encoding 
efficiency. 

Subsequently, the inputted multivalue image data 
is supplied to a block division unit 7. The block 
division unit 7 divides the multivalue image data into 

20 multivalue image data corresponding to a block of a 

predetermined size shown in Fig. 2, for example, a size 
of Hb pixels in the vertical direction and Wb pixels in 
the lateral direction and sequentially outputs the 
divided multivalue image data on a block unit basis. 

25 The outputted multivalue image data is inputted to 

a discrimination unit 1 in Fig. 1. The discrimination 
unit 1 detects the number of gradations which one pixel 
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has, thereby discriminating whether the multivalue 
image data indicates a binary image or a multivalue 
image. The image data discriminated by the 
discrimination unit 1 is also inputted to a switching 
5 unit 6 . 

On the basis of a discrimination result of the 
discrimination unit 1, the multivalue image data 
inputted to the switching unit 6 is outputted to a 
prediction conversion unit 3 if it is the multivalue 

10 image data to be expressed as a binary image and is 

outputted to a discrete wavelet conversion unit 2 if it 
is the multivalue image data to be expressed as a 
multivalue image. That is, a flowing direction of the 
multivalue image data of the switching unit 6 is 

15 switched on the basis of the discrimination result of 
the discrimination unit 1 • 

Predetermined converting processes are performed 
to the multivalue image data inputted to the discrete 
wavelet conversion unit 2 or prediction conversion unit 

20 3. Each of the converting processes will be described 
hereinlater . 

Conversion coefficients after they were converted 
by the conversion units are inputted to an area 
dividing unit 4. The discrimination result for each 
25 block by the discrimination unit 1 is also inputted to 
the area dividing unit 4. 

In the area dividing unit 4, the conversion 
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coefficients which are inputted are compared with a 
predetermined threshold value. On the basis of 
comparison results, the conversion coefficients are 
further sequentially divided into small areas. A 
5 dividing process will be also explained hereinlater. 
The area dividing unit 4 outputs division 
information indicative of a result of the dividing 
process of the block and the actual values and positive 
and negative signs of the conversion coefficients as 
10 bit data. The division information and the bit data 

are inputted to an entropy encoding unit 5 at the post 
stage. 

The entropy encoding unit 5 entropy encodes the 
inputted bit data and outputs final encoded data. 
15 The detailed operation of each block will now be 

described . 

The block division unit 7 divides the inputted 
multivalue image data into blocks of a predetermined 
size and sequentially outputs the multivalue image data 

20 on a divided block unit basis. 

It is now assumed that the multivalue image data 
divided into rectangular areas of a size of Hb pixels 
in the vertical direction and Wb pixels in the lateral 
direction as shown in Fig. 2 is outputted. 

25 The discrimination unit 1 checks each bit of the 

inputted multivalue image data and discriminates 
whether the multivalue image data should be expressed 
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as a binary image (character, diagram, or the like) or 
a multivalue image (natural image such as photograph, 
picture , or the like ) * 

Fig. 3 is a conceptual diagram showing the 
5 operation of the discrimination unit 1 . The inputted 
image data is compared by a comparison unit 1010 and a 
comparison result is inputted to a counter array 1020. 

Each reference value of the comparison unit 1010 
corresponds to a pixel value of the multivalue image 

10 data which is inputted and its number L is set to be 
equal to the maximum level number (for example, 256) 
which can be presumed for the multivalue image data. 

The comparison unit 1010 generates "1" when the 
result of the inputted image data is equal to each 

15 reference value and the result becomes an input to a 
corresponding counter of the counter array 1020. On 
the other hand, the image data inputted to the 
comparison unit 1010 is counted every pixel. At a time 
point when all of the pixels in the relevant block are 

20 compared, the contents of the counter array 1020 are 
outputted to a discrimination unit 1030 and the 
contents of the counter array 1020 are reset. The 
discrimination unit 1030 scans the contents of each 
counter C(i) and counts the number N c of counters of 

25 C(i) * 0 (i = 0, L-l) (1) 

When 

N c = 2 (2) 
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it: is determined -that the image data should be 
expressed as a binary image. When 

N c > 2 (3) 
it is decided that the image data should be expressed 
5 as a mult lvalue image. 

A discrimination result is outputted to the 
switching unit 6 and area dividing unit 4. 

On the basis of the discrimination result inputted 
from the discrimination unit 1, if the multivalue image 
10 data which is inputted should be expressed as a 

multivalue image, the switching unit 6 switches so as 
to output it to the discrete wavelet conversion unit 2. 
If the multivalue image data which is inputted should 
be expressed as a binary image, the switching unit 6 
15 switches so as to output it to the prediction 
conversion unit 3. 

Encoding process as a multivalue image 

Subsequently, a case where the multivalue image 
data which is inputted should be expressed as a 
20 multivalue image, namely, it should be processed as 

multivalue image data as it is (case where the discrete 
wavelet conversion unit 2 is made operative) will be 
described . 

The discrete wavelet conversion unit 2 performs a 
25 discrete wavelet conversion to the inputted multivalue 
image data and divides it into a predetermined number 
of frequency bands (hereinafter, referred to as 
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subbands ) . 

Fig. 4A shows a schematic diagram to execute the 
discrete wavelet conversion. Fig. 4B shows a 
conceptual diagram of subbands which are generated by 
5 the converting process. 

In Fig. 4A, multivalue image data x which is 
inputted passes either a low pass filter HO or a high 
pass filter HI with respect to each of the horizontal 
and vertical directions as shown in the diagram. By 
10 performing a sub-sampling each time the data x passes 
the filter, the data is divided into a plurality of 
frequency bands. 

Fig. 4B shows a processing result that is obtained 
by performing conversions of three stages shown in Fig. 
15 4A to the multivalue image data corresponding to the Wb 
pixels in the lateral direction and the Hb pixels in 
the vertical direction. 

In the embodiment, a size of block shown in Fig. 
4B corresponds to a size (Wb x Hb) of the image divided 
20 into blocks by the block division unit 7. 

It is assumed that each filter which is used in 
the embodiment is a reversible filter such that data 
can be perfectly reproduced at the time of image 
decoding. 

25 For example, a result r obtained by performing 

filtering process by the low pass filter HO and a sub- 
sampling to the multivalue image data x is expressed by 
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the following relational equation (4). A result d 
obtained by performing a filtering process by the high 
pass filter HI and a sub-sampling to the multivalue 
image data x is expressed by the following relational 
5 equation ( 5 ) . 

r(n) = <<(x(2n) + x(2n+l))/2>> (4) 
d(n) = x(2n+2) - x(2n+3) + 

<<(-r(n) + r(n+2) + 2)/4>> (5) 
where, <<X>>: the maximum integer which does not 
10 exceed X 

The discrete wavelet conversion unit 2 
sequentially repeats the filtering process and the sub- 
sampling in the horizontal and vertical directions as 
shown in Fig. 4A as mentioned above and sequentially 
15 divides each block image which is inputted to a 
plurality of subbands. 

Fig. 4B shows a name of each subband obtained by 
Fig. 4A and a spatial positional relation. In each 
subband, a conversion coefficient (frequency component) 
20 corresponding thereto is included. The conversion 

coefficients which are derived by the discrete wavelet 
conversion unit 2 are outputted to the area dividing 
unit 4 at the post stage on a subband unit basis. 
In the embodiment, a subband of LL is first 
25 outputted, subbands of HL3, LH3, and HH3 are outputted, 
subbands of LH2, HL2, and HH2 are outputted, and 
subbands of LH1, HL1, and HH1 are subsequently 
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outputted. By outputting the subbands as mentioned 
above, the frequency components can be sequentially 
decoded from the low frequency component (LL) on the 
decoding side and an outline of the image can be early 
5 recognized . 

In the area dividing unit 4 for inputting the 
conversion coefficients (multivalues ) from the discrete 
wavelet conversion unit 2, only the coefficients having 
values within a certain predetermined range ( in case of 

10 expressing the conversion coefficients of multivalues 

by a bit plane, coefficients existing in a bit plane in 
which the maximum significant coefficient bit indicates 
a value within a predetermined range (which is equal to 
or larger than n,^, which will be explained 

15 hereinlater ) ) are extracted from the conversion 
coefficients and the information of each of the 
extracted coefficients is outputted as a bit data 
string . 

Figs. 5A to 5E show a conceptual diagram for 
20 explaining the operation of the area dividing unit 4. 

For the subbands which are sequentially inputted, 
the area dividing unit 4 converts the conversion 
coefficients in each subband into a bit data string in 
accordance with a procedure shown in a flowchart of 
25 Fig. 5E. Explanation will now be made with reference 
to the f lowchar t . 

First, although sizes of subbands which are 
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Inputted are not always the same size as shown in Fig. 
4B, the inputted subbands themselves are defined as a 
subject block T as shown in Fig. 5A. 

A maximum value c nax for all of the conversion 
5 coefficients c i5 in the subject block T is subsequently 
obtained. Further, n^ is obtained as follows and is 
set to an initial value of a subject bit plane number 
n. 

^ = log 2 (|c max | ) (6) 
10 In step S500, a threshold value Th regarding the 

subject block T is obtained by the following equation 
( 7 ) by using the value of the number n which has been 
set at present. 

Th = 2 n (7) 
15 In step S501, when the present number n is less 

than a predetermined minimum bit plane number n mln , the 
process of the subject block T is finished. 

In steps S502, S503, S504, and S505, with respect 
to the present number n, when the maximum value c oax of 
20 the coefficients in the subject block T is as follows, 
|c ma J 2: Th (8) 
it is determined that there are significant conversion 
coefficients in the subject block T. The processing 
routine advances to step S506. 
25 When the equation (8) is not satisfied, step S503 

follows and bit data "0" indicating that no significant 
conversion coefficient exists in the subject block T is 
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generated . 

In streps S504 and S505, the threshold value Th is 
set to Th/2 and the value of n is reduced by "1" 
(height of bit plane which is used as a threshold value 
5 is reduced by "1"). After that, the processing routine 
is returned to step S501. 

In step S506, bit data "1" showing that the 
significant conversion coefficients exist in the 
subject block T is generated. 
10 In step S507, whether a size of present subject 

block T is equal to or less than a predetermined size 
(two pixels in both of the vertical and lateral 
directions in the embodiment) or not is discriminated. 
As a result of the discrimination, when the size of 
15 subject block T is equal to or less than the 

predetermined size, step S508 follows. When it is 
larger than the predetermined size, step S509 follows. 

In step S508, bit data within a range from the nth 
bit plane to the n^th bit plane corresponding to all 
20 of the coefficients (four coefficients in case of the 

embodiment) in the subject block T and sign bits of the 
coefficients are generated. 

In step S509, the subject block T is further 
divided into four subject blocks T n of a small size. 
25 Processes in step S500 and subsequent steps are 

repeated with respect to each of the subject blocks T n . 
The above procedure will now be described with 
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regard to specific examples. Fig. 5A shows the subject 
block T (each subband in Fig. 4B) which is first 
defined. The subject block T in this case is 
constructed by ( 8 x 8 ) conversion coefficients. 
5 As for the significant conversion coefficients, 

one (a value is "40") significant conversion 
coefficient shown by a hatched region exists in the 
subject block T and has the maximum value in the block. 
The other conversion coefficients are not significant 

10 and their values are equal to "0". 

When the process to sequentially divide the 
subject block T into the subject blocks T m is executed 
in accordance with the procedure of Fig. 5E, it is 
divided in accordance with the order of Fig. 5A Fig. 

15 5B -» Fig. 5C„ so that a bit data string as shown in 
Fig. 5D is outputted. 

The above process relates to the case where n = 5 
and n^ = 3. 

By executing the above process, a bit data string 
20 as shown in Fig. 5D such that the position and value of 
each coefficient can be expressed can be outputted from 
the block comprising a plurality of multivalue 
coefficients . Arrows ( 1 ) , ( 2 ) , and ( 3 ) shown in Fig . 
5D indicate the order by which the bit data is 
25 generated by the process and the order in which it is 

outputted to the post stage. The bits are outputted in 
accordance with the order of the arrows ( 1 ) -+ ( 2 ) 
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(3). 

The above processes will now be described in 
detail hereinbelow. 

At the time point of Fig. 5A, a check is first 
5 made to see whether there is a coefficient in which the 
significant bit exists on the bit plane of the fifth 
bit or not. In this instance , it will be understood 
that only one such coefficient (significant 
coefficient ) exists. 

10 Subsequently, since the position of the 

significant coefficient is not specified in the state 
of Fig. 5A, the subject block T is divided into four 
blocks as shown in step S509. Thus, subject blocks of 
TO to T3 shown in Fig. 5B are decoded. At this time, 

15 bit data "1" showing that the subject block T was 
divided into four blocks is outputted. 

Similar processes are executed with respect to 
each of the subject blocks TO to T3. That is, the 
processes in step S500 and subsequent steps are also 

20 executed for the blocks TO to T3. 

Although the above processes are first executed 
with respect to the subject block TO, after the 
presence of the significant coefficients was decided in 
step S502, the subject block is further divided into 

25 four blocks TOO to T03 . At this time, bit data "1" 

showing that the subject block TO was divided into four 
blocks is generated. 
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Since the block TO is further divided into four 
blocks, the processes from step S500 are executed with 
respect to each of the blocks TOO to T03 prior to 
processing the blocks Tl to T3. 
5 When the above processes are executed with respect 

to the subject block TOO, since a size of subject block 
TOO is equal to ( 2 x 2 ) pixels in step S507, the 
processing routine advances to step S508. Therefore, 
as for the block TOO, the data of the fifth to third 

10 bits corresponding to each coefficient in the block, 
namely, 3-bit data "000", "000", "000", and "101" 
showing the presence or absence of the significant 
coefficient and 1-bit data "0", "0", "0", and "0" 
showing the positive or negative sign of each 

15 coefficient are sequentially outputted in accordance 
with the order shown in the drawing. The above 
procedure corresponds to the arrow ( 1 ) . 

The processes for TOO are finished as mentioned 
above. Subsequently, processes for T01 are executed. 

20 No significant coefficient exists in the third to fifth 
bits in each conversion coefficient in T01. Therefore, 
by repetitively executing the processes in steps S502 
and S503 to S505 to n = 3, 4, and 5, "0" indicative of 
"the absence of the significant coefficient" is 

25 generated until the number n is less than 3. 

Subsequently, processes for T02 and T03 are similarly 
executed, so that "000", "000", and "000" are 
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outputted. The above procedure corresponds to the 
arrow ( 2 ) . 

Since the processes for TO (TOO to T03) are 
finished as mentioned above, Tl to T3 are subsequently 
processed. Since no significant coefficient exists in 
the third to fifth bits in any one of the conversion 
coefficients in Tl to T3, "000", "000", "000" are 
outputted in a manner similar to T01 to T04. The 
procedure corresponds to the arrow ( 3 ) . 

The bit data outputted by the above processes is 
supplied to the entropy encoding unit 5 at the post 
stage. 

Encoding process as a binary image 

Explanation will now be made with respect to the 
case where the multivalue image data which is inputted 
should be expressed as a binary image by the 
discrimination unit 1, namely, it should be processed 
as binary image data (case where the prediction 
conversion unit 3 is made operative). 

When it is decided that the multivalue image data 
which is inputted should be expressed as a binary 
image, the multivalue image data is outputted to the 
prediction conversion unit 3 by the switching unit 6. 
The prediction conversion unit 3 converts the 
multivalue image data which is inputted on a pixel unit 
basis as a binary image into prediction error data. 

Figs. 6A and 6B are conceptual diagrams for 
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explaining -the operation of the prediction conversion 
unit 3. In Figs, 6A and 6B, when the multivalue image 
data x is inputted on a pixel unit basis from the 
switching unit 6, a binarization unit 300 converts it 
5 into binary image data x x of every pixel and outputs, 
A pattern checking unit 301 extracts peripheral 
pixels (10 pixels shown in Fig, 6B, which will be 
explained hereinlater) of the pixel shown by x x 
(referred to as an encoding subject pixel). At the 

10 same time, a template 302 in which all of possible 

combinations of the peripheral pixels and information 
(corresponding to a status S(i), which will be 
explained hereinlater) showing the combinations are 
stored is referred. 

15 Thus, the status S(i) corresponding to the 

extracted actual peripheral pixels is decided and 
outputted to the post stage. The status S(i) is used 
for generation of a prediction value when encoding the 
inputted encoding subject pixel. 

20 Fig. 6B shows a positional relation between the 

present encoding subject pixel x x to be encoded and the 
peripheral pixels which are extracted to generate the 
status S(i). in the diagram, aO to a9 denote pixels 
which have already been encoded. 

25 The status S(i) will now be specifically 

explained. Since the binary image data x x which is 
inputted is image data of one bit of each pixel, each 
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pixel value has either 0 or 1 . Therefore, the number 
of combinations of the possible values of the 
peripheral pixels aO to a9 of the encoding subject 
pixel is equal to 1024. 
5 This state is the status S(i) showing a situation 

of the pixel values of the peripheral pixels of the 
actual encoding subject pixel, "i" denotes that the 
status corresponds to the ith combination among the 
1024 combinations . 

10 In the embodiment, since the binary image data x x 

is inputted to the pattern checking unit 301 by 
completing in the block of (Hb x Wb) pixels, in the 
encoding subject pixels existing at the left edge, 
upper edge, and right edge of the block, there is a 

15 case where the above 10 peripheral pixels do not exist. 
In the embodiment, in order to cope with such a 
situation, the processes are performed on the 
assumption that "0" exists in the peripheral pixels 
which do not actually exist. 

20 The status S(i) is inputted to a prediction unit 

303 . The prediction unit 303 outputs any one of 
prediction values x 2 stored in correspondence to each 
inputted status S(i) to a comparison unit 304. That 
is, the prediction unit 303 is a look-up table. The 

25 prediction value x 2 corresponding to each status S(i) is 
the optimum value obtained by previously analyzing a 
binary sample image. 
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The binary image data x x is also inputted to the 
comparison unit 304. That is, the data x x of a certain 
encoding subject pixel and the prediction value x 2 
obtained by predicting the encoding subject pixel on 
5 the basis of the 10 peripheral pixels are 

simultaneously inputted to the comparison unit 304. 

The comparison unit 304 compares the data x x of the 
encoding subject pixel with the prediction value x 2 , 
generates n 0" as a prediction error e when they 

10 coincide, and generates "1" as a prediction error e 
when they do not coincide. The above processes are 
sequentially executed for the data x x (x) of all of the 
pixels which are inputted. 

Since the block division unit 7 executes the 

15 switching and output of the image data on a block unit 
basis of a predetermined size to the discrete wavelet 
conversion unit 2 or prediction conversion unit 3, the 
process of the prediction conversion unit 3 is 
completed every predetermined size of (Hb x Wb). 

20 Therefore, by constructing the prediction error e 

outputted every block of each predetermined size so as 
to have a relation similar to the actual position of 
each encoding subject pixel, a map E of a size (Hb x 
Wb) is constructed. 

25 The map E obtained by the prediction conversion 

unit 3 is subsequently outputted to the area dividing 
unit 4. The area dividing unit 4 handles the map E in 
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a manner similar "to the subbands outputted from the 
discrete wavelet conversion unit 2, so that the map E 
is processed by using almost similar processing steps. 
The area dividing unit 4 outputs division information 
5 showing a division result when the map E is divided and 
the actual prediction error values and 

positive/negative signs of the conversion coefficients 
as bit data. 

Different from the conversion coefficients in the 

10 subband outputted from the discrete wavelet conversion 
unit 2, the prediction error value in the map E is 
limited to 0 or 1. Therefore, on the basis of a result 
of the discrimination which is inputted from the 
discrimination unit 1, namely, on the basis of 

15 information showing whether the inputted conversion 

coefficients construct the subband which is generated 
from the discrete wavelet conversion unit 2 or 
construct the map E which is generated from the 
prediction conversion unit 3, the area dividing unit 4 

20 switches the area dividing process and a outputting 
method of the bit data string. 

Specifically speaking, as for the conversion 
coefficients, for the case where the subband inputted 
from the discrete wavelet conversion unit 2 is divided 

25 into areas and the bit data string as shown in Fig. 5D 
is outputted, in the case where the map E inputted from 
the prediction conversion unit 3 is divided into areas 
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and the bit data string is outputted, the following 
changes are executed . 

(1) At the time of initialization, the dividing 
subject area T is set to the whole map E and 

5 = 0 (9) 

is set. The subject bit plane number n and the minimum 
bit plane number n Bln are set to 0. 

( 2 ) The block comprising the multivalue conversion 
coefficients is not divided but it is sufficient to 

10 merely divide the block comprising the 1-bit prediction 
error coefficients. Therefore, data indicative of one 
bit of the prediction error is outputted in place of 
outputting three bits indicative of the presence or 
absence of the significant coefficients and one bit 

15 showing the positive/negative sign as shown in Fig. 5D. 
For example, now assuming that T shown in Fig. 5A 
indicates the map E (a hatched area c3 indicates the 
prediction error value "1" and the other areas show 
"0"), bit data strings which are outputted are as 

20 follows in Fig. 5D. 

c0 = cl = c2 = 0 
c3 - 1 

T01 = T02 = T03 = 0 
Tl = T2 = T3 = 0 
25 That is, the bit data string "1110001000000" is 

outputted in accordance with the order of the arrows 
(1), (2), and (3). 
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The construction except for the above two points 
is fundamentally similar to that in case of area 
dividing the multivalue conversion coefficients and its 
description is omitted. 
5 The bit data string outputted as mentioned above 

is encoded by the entropy encoding unit 5 and is 
outputted as final encoded data. 

Data format 

Figs. 7A to 7F show conceptual diagrams of a data 
10 format of encoded data which is generated by the 

encoding as a multivalue image or the encoding as a 
binary image described above. 

Fig. 7A shows a whole construction of the encoded 
data which is finally outputted by the image processing 
15 apparatus of the embodiment. The encoded data is 

constructed by header information and a code stream. 

As shown in Fig. 7B, the header information is 
constructed by: geometry information including the size 
of encoded image, the block size (Hb x Wb) obtained by 
20 dividing the image, and other information which is 
necessary upon decoding; and a level number of the 
discrete wavelet conversion (DWT). 

As shown in Fig. 7C, a code stream is constructed 
by setting the divided block (Hb x Wb) to one unit. A 
25 block header is included in one unit. 

A flag indicating whether the relevant block has 
been encoded as a multivalue image or encoded as a 
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binary image is included in the block header. 

Further, in the case where the block is encoded as 
a mult lvalue image, the number n^ of the highest bit 
plane in which the significant coefficient exists among 
5 the mult lvalue bits showing each conversion coefficient 
in the block is included in the block header. That is, 
the value n^ can be also different every block that is 
encoded as a multivalue image. In the embodiment, n^ 
is common to each block to be encoded as a multivalue 

10 image. However, by constructing such that the 

information of n^ is also included in the encoded data 
in a manner similar to n^, n tnin can be changed every 
block to be encoded as a multivalue image. In the case 
where the block is encoded as a binary image, the 

15 information of n^ and n^ is unnecessary. 

Figs. 7D and 7F show constructions of code streams 
corresponding to each block divided by the block 
division unit 7. 

Fig. 7D shows a construction of the code stream in 

20 the case where the block is encoded as a multivalue 
image. The code stream is further constructed every 
subband of the discrete wavelet conversion. 

Fig. 7E shows a data format corresponding to one 
subblock among the subblocks shown in Fig. 7D. Each 

25 subblock has a similar data format. LL to HHl denote 
the same denominations as those of the subbands shown 
in Fig. 4B. 
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According to the contents in Fig. 7E, two kinds of 
bit streams of a division bit and a coefficient bit 
alternately appear as shown in the diagram. The 
division bit is a bit which is outputted in steps S503 
5 and S506 in the flowchart of Fig. 5E and the 

coefficient bit is a bit which is outputted in step 
S508. 

Fig. 7F shows a construction of the code stream in 
the case where the block is encoded as a binary image. 
10 The code stream has a format such that the division bit 
and a prediction error bit are alternately arranged. 

Decoding process 

A method of decoding the encoded data formed by 
the encoding method of the embodiment will now be 
15 described. 

Fig. 8 is a block diagram showing a decoding 
apparatus for decoding the encoded data formed by the 
encoding method of the embodiment. Explanation will 
now be made in detail hereinbelow with reference to 
20 Fig. 8. 

The encoded data inputted through a transmission 
path or an accumulating media is decoded by an entropy 
decoding unit 13. In a header information analyzing 
unit 8, necessary information such as size of original 
25 image, size of block, and the like is read out and 
stored as information necessary for decoding. 

The block header (refer to Fig. 7C) in the encoded 
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data is read out and whether the blocks which are 
sequentially read out have been encoded as mult lvalue 
images or encoded as binary images is discriminated. A 
discrimination result is outputted to the switching 
5 unit 6. Further, for the block encoded as a multivalue 
image, n^ is read and stored* 

The code streams (refer to Figs. 7C to 7F) read 
out on a block unit basis are inputted to an area 
dividing unit 9 . The area dividing unit 9 is used to 

10 decode data (the subblock comprising the conversion 

coefficients or the map E) just before it is inputted 
to the area dividing unit 4 in Fig. 1 from the division 
bit and coefficient bit by which the relevant block is 
divided upon encoding or the prediction error bit. 

15 The operation of the area dividing unit 9 in the 

case where a certain block shown by the encoded data 
that is inputted has been encoded as a multivalue image 
will now be described hereinbelow with reference to a 
flowchart of Fig. 9F. In the embodiment, it is assumed 

20 that the order when a plurality of subbands are encoded 
( area divided ) on the encoding side has been 
predetermined . 

First, the area of the relevant subband is assured 
from the code stream corresponding to a certain 

25 subband. The area corresponding to the size of this 

subband is set to an initial value of the subject block 
T. All of the coefficients in the area are initialized 
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to 0. The value of n^ which is derived from the block 
header is set to an initial value of the subject bit 
plane n. 

Subsequently, in step S900, when the value of n is 
5 less than the predetermined minimum bit plane number 

n^, the processes of the subject block T are finished. 

In step S901, one division bit is read out from 
the code stream. 

In steps S902 and S903, the value of the read 
10 division bit is checked* When it is equal to 0, 1 is 
subtracted from n and the processing routine is 
returned to step S900. 

In steps S904 and S905, when the size of present 
subject block T is equal to or less than a 
15 predetermined size, for example, the size of 2 pixels 

in the vertical and lateral directions, the coefficient 
bit is read out from the code stream and all of the 
coefficients in the area are decoded. 

In step S906, the present subject block T is 
20 divided into four small areas T n and the same processes 
as those in the processes in step S900 and subsequent 
steps are sequentially executed with respect to each of 
the small areas. 

The above procedure will now be described in 
25 detail with respect to specific examples. Figs. 9A to 
9E show decoding states of the subject block T and code 
streams (bit data strings) regarding the subject block 
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T which are sequentially inputted. 

Arrows ( 1 ) f , ( 2 ) ' , and ( 3 ) 1 shown in Fig . 9E 
indicate the orders in which the bit data strings are 
actually inputted every bit in accordance with the 
5 order of ( 1 ) ' , ( 2 ) ' , and ( 3 ) » . 

In the initial state, as shown in Fig. 9A, all of 
the subject blocks T have been initialized to 0 and are 
not divided. In step S901, the division information 
"1" of first one bit of the arrow (l) 1 is inputted. 

10 Thus, it is recognized that the subject block T 

includes the significant coefficient with respect to 
the present bit plane n. In step S906, the block T is 
divided into four small blocks TO to T3. This 
procedure is shown as a state of Fig. 9B. 

15 A decoding process of TO will now be executed. 

First, since the second bit (division bit "1") of the 
arrow (l) f is inputted in step S901, it is possible to 
decide that there is the significant pixel in the 
conversion coefficients in TO. Therefore, the block TO 

20 is further divided into four small blocks TOO to T03 
(refer to Fig. 9C). 

Since the first small block TO is further divided 
into small blocks here, the decoding process of TOO to 
T03 is preferentially executed. 

25 First, with respect to the block TOO, the 

processes are executed from step S900. In step S901, 
the division bit shown in the third bit (division bit 
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"1") of the arrow (1)' is inputted and it is possible 
to decide that the significant conversion coefficient 
exists in TOO. However, since the size of TOO 
corresponds to the size of two pixels in the vertical 
5 and lateral directions, TOO is not further divided. 

The processing routine advances to step S905 and the 
remaining bit data strings of the arrow ( 1 ) ' are 
inputted. 

Now, since n^ = 5 has been known from the 
10 information included in the encoded data and the value 
of n^ has also previously been known, by which number 
of bits each conversion coefficient is expressed can be 
decided* Therefore, first three bits in the bit data 
strings which are inputted are inputted as values 
15 showing the third to fifth bits of cO and subsequent 

one bit is inputted as a positive/negative sign of the 
conversion coefficient. By this method, a conversion 
coefficient "40" of cO is decoded. Similarly, with 
regard to cl to c3 as well, the data is inputted and 
20 the conversion coefficient "0" is decoded (refer to 
Fig. 9D). 

Since the block TOO has been decoded as mentioned 
above, it will be understood that the bit data strings 
which are subsequently inputted by the arrow ( 2 ) 1 are 
25 used for decoding from T01. 

Since the first three bits which are inputted for 
decoding the block T01 are equal to "000", it will be 
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understood that the conversion coefficient in which " 1" 
exists at the fifth to third bits of T01 does not 
exist. Therefore, the conversion coefficients in the 
block T01 are decoded as 0. Similarly, T02 and T03 are 
5 also processed and the conversion coefficient 0 is 
decoded in the block. 

Further, Tl to T3 are also decoded from the bit 
data string of the arrow ( 3 ) f and the conversion 
coefficient "0" is decoded in the block. 

10 The above operations are also similarly executed 

even in case of decoding the map E encoded as a binary 
image. In this case, however, the above operations 
differ from in that the initial value of T is set for 
the whole block and that ^ is initialized to 0 in a 

15 manner similar to the case of encoding and the bit of 
the positive/negative sign is not included. The 
details of the decoding process are omitted because it 
is sufficient to merely perform the process opposite to 
the encoding process. 

20 The decoded conversion coefficient corresponding 

to each block as mentioned above is outputted to the 
switching unit 6. 

On the basis of information showing whether the 
decoded conversion coefficient which is inputted from 

25 the header information analyzing unit 8 has been 

encoded as a multivalue image or encoded as a binary 
image, the switching unit 6 in Fig. 8 outputs the block 
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of the decoded conversion coefficient: to an inverse 
discrete wavelet conversion unit 10 or an inverse 
prediction conversion unit 11. 

A case where the decoded conversion coefficient 
5 has been encoded as a multivalue image will now be 
described hereinbelow. 

The inverse discrete wavelet conversion unit 10 
combines the subbands which were decoded and inputted 
to the original block construction shown in Fig. 4B and 
10 inversely converts to the value before the filtering 
process is executed in Fig. 4A. Although the inverse 
conversion is realized by repetitively performing the 
up-sampling and the filtering processes of the low pass 
filter and the high pass filter, the filter 
15 coefficients in this instance satisfy the 

reconstructing condition for those used upon encoding. 
For example, for the filters shown in the equations (4) 
and (5), the image data string x is calculated from 
x(2n) = r(n) + <<(p(n)+l )/2>> (10) 
20 x(2n+l) = r(n) - <<(p(n))/2>> (11) 

where, <<X>>: the maximum integer which does not 

exceed X 
by assuming that 

p(n) = d(n-l) - <<(-r(n-l) + r(n+l))/4>> (12) 
25 Thus, the conversion coefficients in the block are 

inversely converted to the multivalue image data and 
are sequentially outputted to a block synthesizing unit 
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12. 

On the other hand, the case where the decoded 
conversion coefficient has been encoded as a binary 
image will now be described. 
5 In this case, each decoded conversion coefficient 

comprising one bit is inputted as a prediction error 
map E. The inverse prediction conversion unit 11 
sequentially inputs the prediction errors e 
constructing the map E and decodes the pixel values by 

10 using the same template as that shown in Figs. 6A and 
6B and the pixel values which have already been 
decoded. The detailed description are omitted here 
since it is sufficient to merely perform the process 
opposite to the foregoing encoding process. The 

15 decoded binary image is converted into the multivalue 
image and becomes multivalue image data. 

The block synthesizing unit 12 synthesizes the 
block of the multivalue image data which is inputted 
from the inverse discrete wavelet conversion unit 10 

20 and the block of the multivalue image data which is 
inputted from the inverse prediction conversion unit 
11, thereby decoding the original image data and 
outputting it. 

According to the above embodiment, in the case 

25 where the area to be encoded as a multivalue image and 
the area to be encoded as binary image data mixedly 
exist in the image shown by the multivalue image data 
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that is inputted, the data can be efficiently encoded. 
That is, a redundancy of a space area which the 
original image has is removed, thereby spatially 
deviating the data. The conversion coefficients which 
5 are derived by the discrete wavelet conversion as a 
mult lvalue image and the prediction conversion as a 
binary image can be concentrated to the positions which 
are spatially deviated. Therefore, the subsequent area 
dividing unit 4 can divide the conversion coefficients 

10 into the partial significant coefficients and the other 
many insignificant coefficients. Thus, an amount of 
bit data which is outputted from the area dividing unit 
4 can be reduced. 

Although the above embodiment has been described 

15 on the assumption that a resolution of the image which 
is decoded from the final encoded data is the same as 
that of the original image, it is also possible to 
perform what is called a progressive encoding such that 
upon decoding, the original image is first decoded at a 

20 low resolution and, subsequently, it can be decoded at 
a high resolution. 

In the following second embodiment, a case of 
constructing the first embodiment so that the 
progressive encoding can be performed will now be 

25 described. 

Even in the second embodiment as well, a 
fundamental construction is similar to that of the 
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first embodiment and the encoding is executed by using 
the construction of Fig. 1* The detailed description 
of each unit, therefore, is omitted here. 

The second embodiment differs from the first 
5 embodiment in a constructing method and a transmitting 
order of the encoded data ( bit data string ) . 

Fig. 10 shows a construction of the bit data 
string corresponding to each subband in case of 
encoding as a multivalue image. 

10 A block (a-1) in Fig. 10 shows 16 blocks (blocks 

of (Wb x Hb) surrounded by broken lines) divided by the 
block division unit 7. Each of the hatched blocks is a 
block which is encoded as a binary image and the 
remaining blocks are blocks which are encoded as 

15 multivalue images. The block (a-1) shows a state after 
it was already discrete wavelet converted. That is, 
the smallest block in each (Wb x Hb) block corresponds 
to the subband LL. 

In the first embodiment, as shown in Fig. 7D, the 

20 subbands LL to HH1 in one block are combined to one 
block. In the embodiment, however, the subbands are 
combined on a unit basis of each level "LL, HL3, LH3, 
HH3 " (assumes level 3), "HL2, LH2, HH2" (assumes level 
2), and "HLl, LH1, HH1 " (assumes level 1) after the 

25 discrete wavelet conversion. For example, in the block 
(a-1), by performing an area dividing operation P, the 
bit data string in which only the subbands at the 
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discrete wavelet conversion level 3 were encoded is 
extracted and outputted (a-2). 

The decoding side which inputs the bit data string 
at level 3 performs an area dividing operation P" 1 at 
5 the time of decoding to each bit data string, decodes 
the coefficients, combines them to one, and constructs 
an area in which only the coefficients at level 3 of 
the discrete wavelet conversion are collected (a-3). 
By performing the inverse discrete wavelet conversion 

10 every block, an image obtained by reducing the original 
image into 1/4 in the vertical and lateral directions 
can be decoded. 

Subsequently, on the encoding side, as shown in 
(b-1), by performing the area dividing operation P, the 

15 bit data string in which only the subbands at level 2 
were encoded is extracted and outputted (b-2). The 
decoding side which inputs the bit data string at level 
2 constructs an area of the conversion coefficients as 
shown in ( b-3 ) . Since the data at conversion level 3 

20 has already been decoded like (a-4), it is used 

together with each block comprising level 2 and the 
inverse discrete wavelet conversion is performed. 
Thus, an image having the resolution or size that is 
twice as large as that of the image decoded in (a-4) 

25 can be decoded (b-4). 

Subsequently, on the encoding side, the bit data 
string in which only the subbands at level 1 were 
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encoded is outputted. On the decoding side as well, by 
executing processes similar to those mentioned above, 
an image having the same resolution or size as that of 
the original image can be finally decoded. 
5 Although the case of encoding as a multivalue 

image has been described above, a case of encoding as a 
binary image can be also progressively encoded. 

Figs. 11A to 11C are diagrams for explaining the 
above method . 

10 Fig. 11A shows the map E comprising the prediction 

error outputted from the prediction conversion unit 3 . 
A result obtained by converting the map E to the bit 
data string by the area dividing unit 4 is shown in 
Fig. 11C. Fig. 11B shows a change in which the initial 

15 subject block T is divided. 

In Figs. 11A and 11B, a portion where the 
prediction error is equal to 1 is shown by a black 
area. A bit data forming method for performing the 
progressive encoding will now be described hereinbelow. 

20 The subject block T in the initial state is 

constructed by (8x8) pixels. Whether the significant 
conversion coefficient (prediction error "1" ) exists in 
the block T or not is discriminated in step S502 in 
Fig. 5E. When it is decided that there is the 

25 significant conversion coefficient, bit "1" showing 

that the block T is further divided is outputted. In 
this state, since the position where the significant 
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conversion coefficient exists in T cannot be 
determined, it is also possible to consider that all of 
the prediction errors in T are equal to "1" at this 
time point. This state is shown by D0(T) at the 
5 leftmost position in Fig. 11B. 

The size of T is checked in step S507. The area 
division is performed in step S509 and T is divided 
into TO to T3. Further, with respect to the blocks TO 
to T3, the processes described in the first embodiment 

10 are executed and the blocks (TO and T3 in this case) 
which were decided to be significant are further 
divided into small blocks. By recursively performing 
the above dividing process, a state of D3 in Figs. 11B 
and 11C, namely, the bit data string to decode the 

15 resolution or size of the original image can be formed. 

In the embodiment, different from the processes 
shown in the flowchart of Fig. 5E, the bit data is not 
actually outputted in steps S506 and S503. In place of 
it, a tree structure of the bits as shown in Fig. 11C 

20 is formed. In Fig. 11C, although there is no branch in 
the portion written as a prediction error, this is 
because since the area reached the minimum size, the 
division is not performed and the bit data itself of 
the prediction error is formed. 

25 The output from the area dividing unit 4 to the 

entropy encoding unit 5 is performed by using a node of 
a predetermined depth in the tree structure as a unit, 
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namely, by using the portion surrounded by a frame in 
Fig. 11C as each unit. Now, assuming that those 
portions are labelled as Dl to D3, a structure of the 
bit data string which is formed is as shown in Figs. 
5 12A to 12D. 

Fig. 12B shows a state when DO corresponding to 
each block divided by the block division unit 7 is 
first outputted to the decoding side. By decoding only 
DO, a map E as shown in the diagram is formed. Each of 
10 the blocks surrounded by broken lines shown here 

corresponds to each block divided by the block division 
unit 7 . 

In case of decoding only DO, each block surrounded 
by the broken lines can express only either black or 

15 white and an image of a very low resolution is decoded. 

Subsequently, Dl is outputted from the encoding 
side. On the decoding side, an image as shown in Fig. 
12C can be decoded on the decoding side from both of Dl 
and DO which was first inputted. 

20 Similarly, by sequentially inputting D2 and D3, 

the decoded image sequentially changes from the low 
resolution to the high resolution and the original 
image as shown in Fig. 12D can be finally decoded. 
As mentioned above, according to the second 

25 embodiment, not only the effect of the first embodiment 
can be obtained but also the progressive encoding such 
that the decoded image on the decoding side is 
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sequentially changed from the low resolution to the 
high resolution can be executed. Even if there is only 
small encoded data on the decoding side, the outline of 
the original image can be grasped. 
5 The invention can be applied to not only the case 

of switching the binary value and multivalue and 
encoding but also the case of switching N values and M 
values and encoding . 

Not only the case of automatically discriminating 

10 the image data by the discrimination unit 1 in Fig. 1 
but also the case of manually switching the switching 
unit 6 is included in the invention. 

The invention can be applied to a system in which 
each unit necessary for the encoding and decoding 

15 processes is connected and used or can be also applied 
as one equipment. 

A third embodiment of the invention will now be 
described hereinbelow with reference to the drawings. 
Fig. 13 shows an image processing apparatus to 

20 execute the third embodiment of the invention. In the 
diagram, reference numeral 101 denotes an image input 
unit; 102 a frame memory to store the whole image 
(picture plane) to be encoded; 103 a block dividing 
circuit; and 104 an arithmetic encoding circuit. 

25 In the embodiment, what is called a binary image 

such that "0" assumes a pixel value of a background and 
"1" assumes a pixel value of a foreground is encoded. 
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The operation of each unit: in the embodiment: will 
now be described in detail hereinbelow. 

First, all of the pixel data (1 or 0) showing the 
image (corresponding to one picture plane) as an 
5 encoding target is sequentially inputted from an image 
input unit in accordance with the raster scanning order 
and is stored into the frame memory 102. 

The image input unit is, for example, an image 
pickup apparatus such as scanner, digital camera, or 
10 the like, an interface of a network line, or the like. 

The block dividing circuit 103 divides the image 
of one picture plane comprising the pixel data stored 
in the frame memory 102 into a plurality of blocks by 
using a characteristic block dividing method such that 
15 blocks of different sizes mixedly exist. 

The block dividing method will now be described in 
detail hereinbelow with reference to the drawings. 

According to the block division, an image of one 
picture plane to be encoded is extracted every block 
20 (explained as (4 x 4) pixels in the embodiment) of a 
predetermined size and this block is further 
recursively divided into four blocks on the basis of a 
pixel distribution, thereby forming small blocks. 

By the bit data string obtained after completion 
25 of the dividing process, a reversible encoding such 

that the arrangement of the background pixels and the 
foreground pixels can be perfectly expressed can be 
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realized, 

A procedure of "the dividing process will be first 
explained. 

Fig. 14 is a flowchart showing a flow of processes 
of the block dividing circuit 103. 

First in step 201 , the image of one picture plane 
stored in the frame memory 102 is read out on a block 
unit basis. That is, prior to performing the block 
division, while the image is previously divided into 
blocks of a predetermined size (4x4 pixels) by a 
block dividing circuit at the post stage, the image 
data is sequentially read out. The block dividing 
circuit 103 further performs the block division every 
read-out block of (4x4) pixels. 

Fig. 15A shows a state of a certain block among 
the blocks of (4 x 4) pixels which are sequentially 
read out. This certain block is referred to as a 
subject block hereinafter and explanation will be made 
by using the subject block. 

First in step 201, a check is made to see if all 
of the pixels in the subject block are set to 
background "0". That is, whether there is no 
foreground "1" in the subject block of (4x4) pixels 
or not is discriminated. 

Thus, if all of the pixels are equal to "0", step 
202 follows. When any one of the pixels is equal to 
"1", step 203 follows. 
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In step 202, bit "0" showing that the original 
block of (4 x 4) pixels is not divided any more is 
outputted as an intermediate code and the dividing 
process of the subject block is finished. 
5 On the other hand, in step 203, bit "1" showing 

that the block is divided is outputted as an 
intermediate code. In step 204, when a size of subject 
block is larger than the predetermined smallest size 
(the block of (2 x 2) pixels is set to the minimum 
10 block in the embodiment), step 205 follows. If NO, 
step 206 follows. 

The set size of minimum block can be manually 
changed . 

In step 205, as shown in Fig. 20, the subject 
15 block is divided into four blocks bO to b3 (in order to 
distinguish from the original block of (4 x 4) pixels, 
these four blocks of (2 x 2) pixels are called 
subblocks). With respect to each of the subblocks b0, 
bl, b2, and b3 formed here as well, the dividing 
20 process is executed in accordance with the procedure 
from step 201 in a manner similar to the foregoing 
block of (4 x 4) pixels. 

In step 206, the pixel values (2x2 pixels in the 
embodiment) in the block are sequentially scanned and 
25 either "1" or "0" is outputted. 

In Fig. 15A, a white pixel indicates a background 
pixel, namely, a pixel of a value "0" and a black pixel 
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shows a foreground pixel, namely, a pixel of a value 
"1". Specific dividing processes of the block dividing 
circuit 103 for the subject block will now be described 
hereinbelow. 

5 First, as for the subject block (refer to Fig. 

15A), since it is determined in step 201 that all of 
the pixels in the block are not equal to "0", "1" is 
outputted as an intermediate code showing the 
information of the block division (step 203). Since it 

10 is decided in step 204 that the size of subject block 
of (4 x 4) pixels is larger than the size of (2 x 2) 
pixels by comparing it with the minimum block size, the 
processing routine advances to step 205. 

In step 205, the subject block is divided into 

15 four subblocks bO, bl, b2, and b3 of (2 x 2) pixels as 
shown in Fig. 15B. The block dividing processes from 
step 201 are further executed with respect to each of 
the subblocks bO, bl, b2, and b3. 

Since all of the pixel values in each of the 

20 subblocks bO, bl, and b2 are equal to "0", the 

processing routine advances to step 202 as a result of 
discrimination in step 201. The intermediate code "0" 
is outputted as information of division for each of the 
subblocks bO, bl, and b2. After that, the dividing 

25 processes of those three blocks are finished. 

On the other hand, as for the subblock b3, since 
all of the pixels in the block are not equal to "0", 
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"1" is outputted as an intermediate code showing the 
information of the block division (step 203). After 
that, since the size of subblock b3 is equal to the 
minimum block size of ( 2 x 2 ) pixels in step 204, the 
5 processing routine advances to step 206. In step 206, 
the block is divided every pixel and four pixel values 
"0", "1", "1", and "1" in the block are also outputted 
as intermediate codes. 

By executing the above dividing processes, a bit 

10 data string "100010111" as shown in Fig. 16 is 

outputted as an intermediate code corresponding to the 
subject block of (4x4) pixels shown in Fig. 15A. It 
can be said that the first, second, third, fourth, and 
fifth bits of the bit data string are division 

15 information indicating whether the block has been 
divided or not. 

When the subject block is decoded from the 
intermediate code on the decoding side, it is first 
known from first bit "1" that the subject block has 

20 been divided into the blocks of ( 2 x 2 ) pixels and it 
is known from subsequent bits "0", "0", and "0" that 
all of the pixels of the first three subblocks bO to b2 
are equal to "0", so that they can be decoded. It is 
known from subsequent bit "1" that only the subblock b3 

25 has been further divided. Since the subblock b3 is the 
minimum block, it is known that subsequent bits "0", 
"1", "1", and "1" are the actual pixel values. All of 
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the pixels are decoded . 

The above intermediate code is a reversible code 
which certainly shows at which position in the subject 
block the significant pixel "1" exists and at which 
5 position "0" exists. 

The intermediate code which is outputted from the 
block dividing circuit 103 is arithmetically encoded by 
the arithmetic encoding circuit 104. In the 
embodiment, QM-Coder is used as a method of arithmetic 

10 encoding. 

When the QM-Coder is used, a procedure for 
adapt ively encoding a binary symbol b of "1" or "0" 
generated in a certain status S in accordance with the 
status S has been described in detail in the 

15 International Standard ITU-T T.81 | ISO/IEC 10918-1 
Recommendation or the like about a still image. 
Therefore, its description is omitted here. As a 
method of classifying the status S, a method of using 
statuses of peripheral pixels of a binary encoding 

20 pixel or the like is known. 

According to the embodiment, however, the 
intermediate code indicative of the information of the 
block division is encoded instead of merely encoding 
the pixel values of the binary image and this method is 

25 a peculiar method different from the well-known method. 
Therefore, the classifying method of the status S will 
be described in detail. 
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In the binary intermediate code which is outputted 
from the block dividing circuit 103, the block division 
information (bit showing whether the block is further 
divided into subblocks or not) and the actual pixel 
5 values (last four bits "0", "1", "1", "1" in Fig. 16) 
in the block after completion of the dividing process 
mixedly exist. The arithmetic encoding circuit 104 
classifies those two bits into different statuses and 
encodes. Particularly, as for the bit indicative of 

10 the block division information, the arithmetic encoding 
according to the detailed status S is executed in 
consideration of the block dividing situation of the 
peripheral blocks. 

A case of arithmetically encoding the bit of the 

15 block division information as a feature of the 

invention will now be described in detail hereinbelow. 

As described in the operation explanation of the 
block dividing circuit 103, the block division 
information shows whether a predetermined block is 

20 divided or not. If the blocks (refer to Fig. 15A) 

which are first sequentially read out are excluded, the 
blocks become the subblocks of the other blocks (blocks 
before the division). In the embodiment, the 
arithmetic encoding is efficiently executed in 

25 consideration of the above point. 

Specifically speaking, in case of arithmetically 
encoding the block division information of the 
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subblock, the status S is classified in accordance with 
the position (the same position as that of any one of 
bO, bl, b2, and b3 ) of the subblock and the number of 
divided subblocks among the other subblocks which are 
5 included in the original block before division and have 
already been divided (for example, bO in case of the 
subblock bl; bO and bl in case of the subblock b2; bO 
to b2 in case of the subblock b3; none in case of the 
subblock bO ) • 

10 For example, when the block division information 

("0" in the embodiment) of the subblock bO is 
arithmetically encoded, the other subblocks which are 
included in the block (subject block in Fig. 15A in the 
embodiment) before it is divided into this subblock and 

15 have already been divided do not exist. Therefore, the 
block division information of the subblock bO is 
arithmetically encoded as a status S(l). A status S(n) 
indicates the nth status. 

When the block division information ("0" in the 

20 embodiment ) of the subblock bl is arithmetically 

encoded, the other subblocks which are included in the 
block (subject block in Fig. 15A in the embodiment) 
before it is divided into this subblock and have 
already been divided are only one subblock bO. 

25 Therefore, the statuses can be classified into two 

statuses S(2) and S(3) indicating whether the subblock 
bO has been divided or not (namely, whether the block 
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division information of bO is equal to "1" or "0"). 
Thus, when the block division information of the 
subblock b2 is arithmetically encoded, the arithmetic 
encoding is executed on the basis of either one of the 
5 statuses S(2) and S(3). 

When the block division information ("0" in the 
embodiment) of the subblock b2 is arithmetically 
encoded, the other subblocks which are included in the 
block (subject block in Fig. 15A in the embodiment) 

10 before it is divided into this subblock and have 

already been divided are the two subblocks bO and bl . 
Therefore, the statuses can be classified into four 
statuses S(4) to S(7) showing whether the subblocks bO 
and bl have been divided or not (namely, whether the 

15 block division information of bO and bl is equal to "1" 
or "0" ). Consequently, when the block division 
information of the subblock b2 is arithmetically 
encoded, the arithmetic encoding is performed on the 
basis of any one of the statuses S(4) to S(7). 

20 When the block division information ("1" in the 

embodiment) of the subblock b3 is arithmetically 
encoded, the other subblocks which are included in the 
block (subject block in Fig. 15A in the embodiment) 
before it is divided into this subblock and have 

25 already been divided are the four subblocks bO to b3. 
Therefore, the statuses can be classified into eight 
statuses S(8) to S(15) showing whether the subblocks bO 
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to b3 have been divided or not: (namely, whether the 
block division information of bO to b3 is equal to "1" 
or "0"). Consequently, when the block division 
information of the subblock b3 is arithmetically 
5 encoded, the arithmetic encoding is performed on the 
basis of any one of the statuses S(8) to S(15). 

In the statuses S(l) to S(15) which were 
individually divided as mentioned above, the 
intermediate codes are sequentially arithmetically 

10 encoded by the QM-Coder. In the QM-Coder, an 

appearance probability of the next symbol is presumed 
from the encoded symbol string every status S and the 
encoding is executed in accordance with the appearance 
probability of the symbol in each status. 

15 The encoding of the binary image data can be 

efficiently executed by the above processes. 

Although the system of encoding the binary image 
data has been provided in the above third embodiment, 
the invention is not limited to it. That is, as for a 

20 point that the efficient arithmetic encoding is 
executed in consideration of the image dividing 
process, particularly, it is not always necessary to 
perform the encoding of the binary image data. The 
invention can be also applied to the case of encoding 

25 multivalue image data. Such a modification will now be 
described as a fourth embodiment of the invention 
hereinbelow . 
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Fig. 17 shows a block diagram of the fourth 
embodiment according to the invention. In the diagram, 
reference numeral 501 denotes an image input unit to 
input multivalue image data of one picture plane and 
5 502 indicates a frame memory to store the multivalue 
image data of one picture plane. In a manner similar 
to the third embodiment, the image input unit 501 is, 
for example, an image pickup apparatus such as scanner, 
digital camera, or the like, an interface such as a 

10 network line, or the like. 

Reference numeral 503 denotes a discrete wavelet 
conversion circuit to discrete wavelet convert 
multivalue image data; 504 a subband extraction circuit 
to sequentially extract a plurality of subbands 

15 generated by the discrete wavelet conversion; 505 a 

block dividing circuit for performing a process similar 
to that of the block dividing circuit 103 in Fig. 1; 
and 506 an arithmetic encoding circuit. 

In the embodiment, explanation will be made on the 

20 assumption that monochromatic multivalue image data of 
8 bits is encoded. However, the invention is not 
limited to such an example. A case where the invention 
is applied to a color multivalue image encoding such 
that 8-bit multivalue image data is encoded as a 

25 luminance L and 8-bit multivalue image data is encoded 
as chromaticities a and b is also incorporated. The 
invention is not limited to the image data but the 
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invention can be also applied to a case of encoding 
multivalue information showing the status of each pixel 
of an image area. 

First, multivalue image data to be encoded is 
5 inputted from the image input unit 501 and stored into 
the frame memory 502. The discrete wavelet converting 
circuit 503 reads out the multivalue image data of one 
picture plane stored in the frame memory 502 every 
block of a predetermined size and sequentially executes 
10 the discrete wavelet conversion. By the discrete 

wavelet conversion, the original block is divided into 
a predetermined number of frequency bands (hereinafter, 
called subbands) as shown in Figs. 18A to 18C. The 
discrete wavelet conversion for image data string x(n) 
15 in the embodiment is executed by the following 
equations . 

r(n) = <<(x(2n) + x(2n+l))/2>> 
d(n) = x(2n) - x(2n+l) 
where, <<X>>: the maximum integer which does not 
20 exceed x 

r(n), d(n): conversion coefficients 
r(n): low frequency component 
d(n): high frequency component 
Although the above conversion equations relate to 
25 one-dimensional data, by sequentially executing the 

conversion in the horizontal direction and the vertical 
direction, the original block can be divided into four 
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subbands of LL, HL, LH, and HH as shown in Fig- 18A. 

Further, by again performing the discrete wavelet 
conversion to the LL component, the block is divided 
into seven subbands as shown in Fig. 18B. The number 
5 of times of execution of the two-dimensional discrete 
wavelet conversion is called a level number. It is 
assumed that each subband is expressed and specified as 
shown by LH3 ( LH component at level 3) by adding the 
level number to LL, HL, LH, and HH showing the 

10 components. In the embodiment, the conversion at level 
4 is performed and the block is divided into 13 
subbands shown in Fig. 18C. 

The subband extraction circuit 504 sequentially 
extracts only specific bands such as LL4, HL4, LH4, 

15 HH4, HL3, LH3, HH1 from the conversion coefficient 

data divided into 13 subbands by the discrete wavelet 
conversion circuit 503 in accordance with the order 
from the low frequency band and outputs them to a 
quantization circuit 507 at the post stage. 

20 In the quantization circuit 507, the values of all 

of the conversion coefficients in each of the subbands 
which are sequentially outputted by the subband 
extraction circuit 504 are divided by a predetermined 
quantization coefficient, thereby performing the 

25 quantization. In the embodiment, 2 is used as a 
quantization coefficient. 

The block dividing circuit 505 recursively 
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executes the block dividing process similar to that in 
the third embodiment with reference to the values of 
the conversion coefficients in each subband quantized 
by the quantization circuit 507. This method will be 
5 explained hereinlater. 

As a result of division mentioned above, the 
intermediate code is formed by outputting the 
information showing the block division and a value 
(positive/negative signs and the absolute value) of the 

10 conversion coefficient of the block whose size is equal 
to or smaller than the predetermined minimum size 
(assumes (2x2) pixels in the embodiment). 

The block dividing process in the embodiment will 
now be described. 

15 In the block division of a binary value in the 

third embodiment, whether the block as a target is 
divided or not is determined on the basis of the 
presence or absence of the significant pixel "1" in the 
block. However, the deciding method is almost the same 

20 in terms of this point. Specifically speaking, whether 
the block is further divided into four subblocks or not 
is determined by checking whether a pixel of a certain 
predetermined density (bit plane of a predetermined 
height) or more exists in the block or not. In case of 

25 the block division of a binary value, when the size of 
subblock after the repetition of the division becomes 
the minimum size, the actual pixel values "1" and "0" 
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in the subblock are generated. In the embodiment, 
however, multivalue bit data is outputted. Explanation 
will now be made hereinbelow with reference to the 
drawings . 

5 Fig. 19 shows a flow of dividing process of a 

multivalue block which is executed by the block 
dividing circuit 505. Processes of the block dividing 
circuit 505 will now be sequentially explained 
hereinbelow with reference to the diagram. 

10 Prior to the block dividing process by the block 

dividing circuit 505, an initial value of the number n 
of the subject bit plane is set. It is assumed that 
the bit plane number is set to 0 for the LSB plane and 
increases one by one toward the MSB plane. 

15 The initial value of the subject bit plane number 

n is set to the theoretical maximum bit number 
necessary to express an absolute value of the 
conversion coefficient in the subband inputted to the 
block dividing circuit 505. This number is determined 

20 by the number of bits per pixel of an image to be 

encoded, the number of bits after completion of the 
wavelet conversion, and further, the number of bits 
after the quantization. 

In case of the embodiment, the multivalue image to 

25 be encoded is expressed by 8 bits per pixel. The 
numbers of bits of the absolute values of the 
conversion coefficients which are derived by the 
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converting process of the discrete wavelet conversion 
circuit 503 differ depending on the components and are 
set to 8 bits for the LL, HL, and LH components and to 
9 bits for the HH component. 
5 Since the subband is further divided by the 

quantization coefficient 2 by the quantization circuit 
507, the initial value of n is set to 7 bits for the 
LL, HL, and LH components and to 8 bits for the HH 
component . 

10 The block dividing circuit 505 first sets the 

whole subbands extracted by the subband extraction 
circuit 504 to a processing target block (subject 
block). First in step 701, the maximum value C max of 
the absolute values of the coefficients in the subject 

15 block is obtained and is compared with a threshold 
value 2 n -l which is determined from the number n of 
subject bits. When C max is larger than the threshold 
value, step 705 follows. If NO, the processing routine 
is returned to step 702. 

20 That is, this is equivalent to that the absolute 

values of the conversion coefficients in the processing 
subject block are natural binary expressed and divided 
into bit planes and whether all of the subject bit 
planes are equal to "0" or not is discriminated. In 

25 step 702, "0" showing that the block is not divided in 
the relevant bit plane is outputted as an intermediate 
code as information of the block division. 
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In s-tep 703, the value of n is examined. When n 
is larger than 0, the processing routine advances to 
step 704. If N0 A the processes about the subject block 
are finished. In step 704, the value of n is reduced 
5 by "1" and the processing routine is returned to step 
701. 

In step 705, the intermediate code "1" showing 
that the block has been divided is outputted as 
information of the block division. In step 706, the 
10 size of subject block is compared with the 

predetermined minimum block size. When the subject 
block size is larger than the minimum block size, step 
707 follows. If NO, the processing routine advances to 
step 708. 

15 In step 707, the subject block is divided into 

four subblocks and the initial value of the subject bit 
plane number is set to the present value of n with 
respect to each of the subblocks. The processes from 
step 701 are executed. 

20 In step 708, the absolute values are natural 

binary expressed by n bits and are outputted from the 
upper bit one by one in accordance with the raster 
scanning order with regard to the coefficients in the 
subject block. If the coefficient value is not equal 

25 to "0", the positive/negative sign ("0" if "+" ; "1" if 
"-" ) is also outputted. 

By the above processes, each of the subbands 
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inputted to the block dividing circuit 505 is converted 
into the intermediate code and outputted. 

The arithmetic encoding circuit 506 arithmetically 
encodes the intermediate codes which are output ted from 
5 the block dividing circuit 505 and outputs the final 

encoded data. In the embodiment as well, it is assumed 
that the arithmetic encoding is performed by using the 
QM-Coder in a manner similar to the third embodiment. 

The details of the processing procedure of the QM- 

10 Coder are also omitted here in a manner similar to the 
third embodiment and only the classifying method of the 
status S which is used for arithmetic encoding will now 
be described. The block division information (bit 
showing whether the block is divided or not ) and the 

15 multivalue conversion coefficient value (bits showing 
the absolute values and positive/negative sign) are 
included in the intermediate code which is outputted 
from the block dividing circuit 505. Those information 
is arithmetically encoded on the basis of the 

20 individual statuses S in a manner similar to the third 
embodiment . 

Further, in the embodiment, when the bits to 
express the absolute values of the conversion 
coefficients are arithmetically encoded, the status is 
25 classified on the basis of the bit plane number to 

which the bit to be encoded belongs. The status about 
the block division information is also finely 
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classified. The block is arithmetically encoded in 
consideration of a fact that except for the subband as 
a first division target, the block becomes the 
subblocks of the other blocks (original block before 
5 division). 

Specifically speaking, in case of arithmetically 
encoding the block division information of the 
subblock, the status S is classified in accordance with 
the position (the same position as that of any one of 

10 bO, bl, b2, and b3 ) of the subblock and the number of 
subblocks among the other subblocks which are included 
in the original block before the division and have 
already been divided (for example, bO in case of the 
subblock bl; bO and bl in case of the subblock b2; bO 

15 to b2 in case of the subblock b3; and none in case of 
the subblock bO ) . 

For example, when the block division information 
of the subblock bO is arithmetically encoded, the other 
subblocks which are included in the block before it is 

20 divided into this subblock and have already been 

divided do not exist. Therefore, the block division 
information of the subblock bO is arithmetically 
encoded as a status S(l). The status S(n) indicates 
the nth status. 

25 When the block division information of the 

subblock bl is arithmetically encoded, the other 
subblocks which are included in the block before it is 
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divided into this subblock and have already been 
divided are one subblock bO. Therefore, the statuses 
can be classified into two statuses S(2) and S(3) 
showing whether the subblock bO has been divided or not 
5 (namely, whether the block division information of bO 

is equal to "1" or "0"). Thus, when the block division 
information of the subblock b2 is arithmetically 
encoded, the arithmetic encoding is executed on the 
basis of either one of the statuses S(2) and S(3). 

10 When the block division information of the 

subblock b2 is arithmetically encoded, the other 
subblocks which are included in the block before it is 
divided into this subblock and have already been 
divided are two subblocks bO and bl. Therefore, the 

15 statuses can be classified into four statuses S(4) to 
S(7) showing whether the subblocks bO and bl have been 
divided or not (namely, whether the block division 
information of bO and bl is equal to "1" or "0"). 
Thus, when the block division information of the 

20 subblock b2 is arithmetically encoded, the arithmetic 
encoding is executed on the basis of any one of the 
statuses S(4) to S(7). 

When the block division information of the 
subblock b3 is arithmetically encoded, the other 

25 subblocks which are included in the block before it is 
divided into this subblock and have already been 
divided are four subblocks bO to b3. Therefore, the 
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statuses can be classified into eight statuses S(8) to 
S(15) showing whether the subblocks bO to b3 have been 
divided or not (namely, whether the block division 
information of bO to b3 is equal to "1" or "0"). Thus, 
5 when the block division information of the subblock b3 
is arithmetically encoded, the arithmetic encoding is 
executed on the basis of any one of the statuses S(8) 
to S(15). 

In the statuses S(l) to S(15) which were 
10 individually classified as mentioned above, the 

intermediate codes are sequentially arithmetically 
encoded by the QM-Coder. In the QM-Coder, an 
appearance probability of the next symbol is presumed 
from the encoded symbol string every status S and the 
15 encoding is executed in accordance with the appearance 
probability of the symbol in each status. 

By the above processes, the encoding of the 
multivalue image data can be efficiently executed. 
The invention is not limited to the foregoing 
20 embodiments. For example, in the fourth embodiment, 
although the filter of two taps has been used in the 
discrete wavelet conversion, another filter of a larger 
number of taps can be also used. Although the QM-Coder 
has been used as an arithmetic encoding method, another 
25 arithmetic encoding such as Q-Coder, CJ-Coder, or the 
like can be also used. 

The system for entropy encoding the intermediate 
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code is not limited to the arithmetic encoding. For 
example, a Huffman encoding such that the method is 
switched in accordance with the status or the like can 
be also used. 

5 Although the third embodiment has been described 

on the assumption that an image of one picture plane to 
be encoded is a mere binary image, the invention is not 
limited to it. That is, the invention also 
incorporates a case where it is applied when position 
10 information of one picture plane showing the position 
"1 M of a photograph area and the position "0" of a 
character /diagram area in one picture plane is encoded 
in place of encoding the binary image of one picture 
plane. 

15 Although the fourth embodiment has been described 

on the assumption that the image of one picture plane 
to be encoded is a mere mult lvalue image, the invention 
is not limited to it. That is, the invention also 
incorporates a case where it is applied when multivalue 

20 identification information for each position in one 
picture plane is encoded in place of encoding the 
multiimage of one picture plane. 

The invention can be applied as a part of a system 
comprising a plurality of equipment (for example, a 

25 host computer, interface equipment, a reader, a 

printer, and the like) or can be also applied to a part 
of an apparatus comprising one equipment (for instance, 
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a copying apparatus, a facsimile apparatus). 

The invention is not limited to an apparatus and a 
method for realizing the foregoing embodiments. A case 
where program codes of software to realize the 
5 embodiments are supplied to a computer (CPU or MPU) in 
the system or apparatus and the computer of the system 
or apparatus makes the above various devices operative 
in accordance with the program codes, thereby realizing 
the embodiments is also incorporated in the scope of 

10 the invention. 

In this case, the program codes themselves of the 
software realize the functions of the embodiments. The 
program codes themselves and means for supplying the 
program codes to the computer, specifically speaking, a 

15 storing medium in which the program codes have been 
stored are also incorporated in the scope of the 
invention . 

As such a storing medium to store the program 
codes, for example, it is possible to use a floppy 

20 disk, a hard disk, an optical disk, a magnetooptic 

disk, a CD-ROM, a magnetic tape, a non-volatile memory 
card, an ROM, or the like. 

The program codes are incorporated in the scope of 
the invention in not only a case where the functions of 

25 the embodiments are realized by controlling the various 
devices by the computer in accordance with only the 
supplied program codes but also a case where the 
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embodiments are realized in cooperation with the OS 
(Operating System) by which the program codes operate 
on the computer, another application software, or the 
like. 

5 Further, a case where the supplied program codes 

are stored into a memory provided for a function 
expansion board of a computer or a function expanding 
unit connected to the computer and, after that, a CPU 
or the like equipped for the function expansion board 

10 or function expanding unit executes a part or all of 

the actual processes on the basis of an instruction of 
the program codes and the embodiments are realized by 
those processes is also incorporated in the scope of 
the invention. 

15 The present invention is not limited to the 

foregoing embodiments but many modifications and 
variations are possible within the spirit and scope of 
the appended claims of the invention. 



